Automatic system for measuring spacing, depth, and geolocation of seeds

ABSTRACT

A system for measuring real-time seed placement of seeds, such as corn seeds, during planting is provided. In certain embodiments, the sensing and measurement (SAM) system comprises various elements selected from the group consisting of a high-speed camera, light-section sensor, potentiometer, GPS unit, data acquisition system, and a control computer. The SAM system measures seeding depth, seed spacing, and geo-location of the seed. It is mounted on a planter row unit located in between the closing wheels and the gauge wheels with the camera and light section sensor directly facing the furrow.

RELATED APPLICATION

This application claims the benefit of U.S. Provisional PatentApplication No. 62/879,169, filed Jul. 26, 2019, which is incorporatedby reference herein in its entirety.

BACKGROUND OF THE INVENTION Field of the Invention

Proper seed placement during planting is critical to achieve the maximumpotential yield of crop. While uniform plant spacing and seeding depthare often used by corn growers to determine the performance of precisionplanters, these parameters can be influenced by other factors which arenot machine related such as germination percentage, diseases, andvarious soil properties. Currently, the ideal way to determine planterperformance is to manually measure seed to seed spacing and seedingdepth. However, this process is laborious, difficult, and prone to humanerrors. The present invention is directed toward an automatedmeasurement system that is capable of measuring seed spacing and seedingdepth and provide geo-location of all planted seeds.

Description of the Prior Art

Corn is one of the primary agricultural commodities in the U.S. whichaccounts for more than 27% of the total cropland harvested (USDA-ERS,2017). Total production was 15.1 billion bushels in 2016 equivalent toaverage operating costs of more than $41 billion spent by corn growers(USDA-ERS, 2017). However, cash income from corn sales in 2017 isexpected to be 0.7% or over $0.3 billion lower than in 2016 and has beencontinuously declining since 2013. In addition, it is projected thatacres planted for corn will be reduced by over 9% in the next 10 years(USDA, 2017b) which also results in a projected decrease in cornproduction (USDA, 2017a). With high cost of production, continuousdecline of corn receipts (USDA-ERS, 2017), and projected reduction inproduction area and yield, more growers are relying on precisionagriculture technologies to cut costs, maximize land area, and improveyield to sustain farming productivity.

Planting is one of the most critical components in agriculturalproduction, which can have a major influence on potential yield. Properplacement provides seeds the ideal environment for germination andgrowth. As such, uniform plant spacing and seeding depth are two of themost important parameters to be achieved during planting. Theseparameters preferably result in a final plant population with thedesired plant density and uniform emergence. Studies have shown theinfluence of multiple plants, non-uniform plant spacing, delayedemergence and uneven seeding depth on grain yield. Khim Chim et al.(2014) conducted a controlled experiment to evaluate effects of varyingplant spacing and plants per hill on corn yield. Results suggestedhigher grain yield at narrower uniform plant spacing with one plant perhill. However, wider uniform plant spacing resulted in a higher yieldwhen the number of plants per hill is increased. Likewise, non-uniformplant spacing caused by multiple or missing plants measured by thespacing standard deviation had varying effects on the yield. Nielsen(2006) suggested significant difference on yield across the differentplant spacing variability (PSV) treatments. An average of 1.7 bushelsper acre yield loss is reported for every inch increase in standarddeviation (SD) of plant spacing. Similar results were reported by Krallet al. (1977) where yield continuously decreased with increasing spacingvariability at two different study areas. Previous studies showed thatreducing the spacing standard deviation by one inch could result in anaverage yield increase of 3.4 bu/acre (Doerge et al., 2002) and 6.3bu/acre when spacing standard deviation was reduced by 2 inches(Nielsen, 2001). Thus, improving planter performance by reducing plantspacing variability is important in increasing yield. Furthermore,variability in seeding depth can affect emergence. Knappenberger et al.(2014) reported that emergence of corn was significantly correlated withseeding depth where deeper seeding depth resulted in higher emergencedue to availability of moisture and warmer soil temperatures which arefavorable conditions for seedling emergence. Grain yield was affectedwhen seeds emerged unevenly. Thomison et al. (2012) conducted a two-yearstudy on effects of seeding depth on yield of corn and showed a 13% to40% yield difference between shallow and deep planting depths. Observedyield effects for the shallow planting depths were due to reduced finalplant population which might be caused by slow and uneven emergence.

While uniform plant spacing and seeding depth are often used by corngrowers to determine the performance of precision planters, theseparameters can be influenced by other factors, which are not machinerelated, such as germination percentage, diseases, and various soilproperties. The ideal way to determine planter performance is to measureseed to seed spacing (Nakarmi et al., 2012) and ability to maintain aconsistent actual planting depth during planting (Anonymous, 2015).However, seed spacing and depth during planting can only be measured bymanually digging the soil furrow. This process is labor intensive and isprone to measurement errors. Post emergence, seed spacing can bemeasured through emerged plants whereas seeding depth measurement isperformed by manually digging emerged plants and measuring the distanceof the seed to the ground level. Each year, agronomists, serviceprofessionals, producers, and engineers manually measure seed spacingand depth on a large amount of plants to validate the accuracy of theplanting systems. Simply digging a sampling of plants is not sufficientto represent whole field variation (e.g., thousands of plants). Thus,automating the process is important to improve the effectiveness andefficiency of planting systems, provide real-time feedback to the systemoperator, gather field-wide planting data, and reduce labor requirementsand susceptibility to errors.

SUMMARY OF THE INVENTION

According to one embodiment of the present invention a seed plantingassembly is provided comprising a seed planting device configured tocreate a furrow in the ground, deposit a seed into the furrow, and closethe furrow. A camera is attached to the seed planting device andconfigured to capture an image of the seed in the furrow prior toclosing of the furrow. A GPS unit is attached to the seed plantingdevice and located in vertical alignment with the camera and operable todetect the geographic coordinates of the center of the image. Aprocessor is also included that is operable to analyze consecutiveimages captured by the camera and stitch the images together therebyforming a stitched image comprising at least two adjacent seeds withinthe furrow. Embodiments of the present invention may also comprise alight section sensor positioned to be in facing relationship to thefurrow and operable to detect the deepest portion of the furrowcontained within the image captured by the camera and the groundadjacent to the furrow. The assembly may also comprise a potentiometermounted on the seed planting device and configured to provideinformation to the processor corresponding to the vertical displacementof the seed planting device during planting operations and to calculatea unit increase or decrease in the measuring position of the lightsection sensor. The assembly may also comprise a light source mounted onthe seed planting device and configured to illuminate the furrow andseeding during image capture by the camera.

According to another embodiment of the present invention there isprovided a method of measuring planting characteristics of seeds. Themethod comprises creating a furrow in the ground and depositing a firstseed within the furrow. A camera is used to capture a first imagecomprising the first seed within the furrow. A GPS unit positioned invertical alignment with the camera is used to detect and record thegeographic coordinates of the center of the first image. A second seedis deposited within the furrow. The camera is used to capture a secondimage comprising the second seed within the furrow. The GPS unit is usedto detect and record the geographic coordinates of the center of thesecond image. A processor is used to stitch the first and second imagestogether to form a stitched image, the stitched image comprising thefirst and second seeds. The processor is used to analyze the stitchedimage to determine the number of pixels between the first and secondseeds and to convert the number of pixels into a basic unit ofmeasurement using a pixel-to-distance calibration factor therebydetermining the spacing between the first and second seeds. In certainembodiments of the present invention, the method may also comprise usingthe geographic coordinates of the centers of the first and second imagesto determine the geographic coordinates of the first and second seeds.The method may also comprise using a light section sensor positioned infacing relationship to the furrow to detect the deepest portion of thefurrow contained within the first and second images captured by thecamera and the ground adjacent to the furrow, and using the processor tocalculate the difference between the deepest portion of the furrow andthe ground adjacent to the furrow to determine a depth of the seed inthe ground. The method may also comprise using a potentiometer mountedon a device that is creating the furrow and depositing the seeds toprovide information to the processor corresponding to a verticaldisplacement of the seed planting device during planting operations andto calculate a unit increase or decrease in the measuring position ofthe light section sensor.

Embodiments of the present invention enable data to be generated thatprovides an automated and accurate assessment of planting operations ona field-wide scale. The data generated includes a precise count of thenumber of seeds planted in a particular field or area, and a number ofcharacteristics associated with the planted seeds, including instant (ornearly instant) determination of seed spacing, seed depth, and thegeolocation of each seed. This data can be provided substantially inreal-time to the operator of the planting equipment so that adjustmentscan be made to the planting equipment as soon as an unfavorablecondition is detected to ensure maintenance of optimal plantingcharacteristics for the seeds and ground in which the seeds are placedthroughout the planting operation.

In addition to the above, the data generated can be used to generate ahighly accurate map of the field in which the seeds are planted, as thegeolocation of every seed is known. This highly accurate map can be usedto guide autonomous farming machinery performing work in the field, suchas delivering fertilizer or pesticide. Knowing the geolocation of eachseed permits autonomous equipment to readily differentiate weeds fromthe growing crops and to apply herbicides to the weeds and avoidapplication of herbicides to the growing crops. Also, routes taken bythe autonomous equipment through the fields can be optimized by knowingthe geolocation of each seed in order to avoid damage to the plants.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 schematically depicts a seed planting assembly according to anembodiment of the present invention and illustrates the alignment of theGPS unit and high-speed camera on the row unit;

FIG. 2 is a schematic depiction of keypoint detection in a stack of DoGimages;

FIG. 3 is a diagram of the framework of the spatial calibrationalgorithm;

FIGS. 4A and 4B depict two consecutive photographs of (a) the referenceimage, and (b) the target image with the overlap between the twoillustrated by the dashed-window;

FIG. 5 is the generated stitched image using the photographs of FIGS. 4Aand 4B;

FIG. 6A is a photograph of the known seed spacing taken manually;

FIG. 6B illustrates the special calibration of the stitched image andthe calibration value;

FIGS. 7A and 7B depict another two consecutive photographs to bestitched together for seed spacing measurement;

FIG. 7C is the stitched image of FIGS. 7A and 7B;

FIG. 8A is a photograph of the known seed spacing taken manually;

FIG. 8B illustrates measured seed spacing using the stitched image;

FIG. 9 is a chart of the seeding depth during planting based upon sensormeasurements (recorded) and actual measured depths (measured);

FIG. 10A depicts a sample image; and

FIG. 10B depicts the GPS coordinates of the seed in FIG. 10A.

While the drawings do not necessarily provide exact dimensions ortolerances for the illustrated components or structures, the drawingsare to scale with respect to the relationships between the components ofthe structures illustrated in the drawings.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Placing seeds at the desired depth and spacing consistently and withinschedule is a challenge for growers during planting season. Row cropplanters have become bigger and more sophisticated. Technologicaladvancements with row crop planters allows planting operations to beaccomplished more precisely and faster than ever before therebyproviding growers the opportunity to maximize yield over a wide range ofoperating conditions. As the width of the planter toolbar increases, sodoes the variability on seed placement across all row units.Understanding how planting technologies respond to field heterogeneitywill provide growers the decision tools to maximize planter performancebased on planting requirements of their field.

Downforce technology in row crop planters has progressed from mechanicalsprings to hydraulic cylinders aiming to accurately manage weight actingon individual row units. Ideal downforce prevents soil compaction andability to place seeds at the right depth with the right soil moistureand temperature providing the ideal seed-to-soil contact leading toproper seed germination and plant development. Field conditions can varybetween plots of land and have been shown to cause variability inreal-time gauge wheel load (GWL) of individual row units which couldpotentially affect planting performance. Soil strength and moisture canbe indicated by soil texture which can be defined as the ability of thesoil to hold water. Coarse-textured or sandy soils have a lower capacityto hold moisture and requires lower resistance for soil penetration. Onthe other hand, fine-textured or clay soils have higher water holdingcapacity and higher penetration resistance. Planting on soil withvarying soil texture requires different levels of downforce to overcomesoil resistance in creating the seed furrow. Actual planting operationsrevealed that fixed downforce setting resulted in significant areaswhere the row unit applied too much or less than 0 GWL. Too much GWLcould cause the seeds to be planted too deep or create furrow sidewallcompaction restricting seed emergence and affecting plant development.Less than 0 GWL suggests uncertain seeding depth due to potential lossof contact between the gauge wheels and the ground surface.Understanding gauge wheel variability and its impact on planteroperation is critical for both selecting the type and resolution ofdownforce control systems for planters.

Most manufacturers started by manufacturing pneumatic downforce controlsystems with section control. However, more recently most manufacturershave moved to adopting hydraulic downforce control on a row-by-rowbasis. There is a verifiable need to vary levels of downforce across arange of machine and field operating conditions. Selecting the idealdownforce at planting is important at it defines how plants will developthroughout the whole growing season. Selection of incorrect downforcecan impact the accurate seeding depth without the operator knowing itfrom the cab display, although the monitor would indicate correctmanagement of user selected target downforce.

Seeding placement and machine dynamics during planting could impactyield variation, and possibly produce significant row-to-row yieldvariations. One or more embodiments of the present invention may providedirect information on the most critical seed placement parameters. Aseeding depth sensing system could provide direct feedback to the userand to fine tune machine control to correctly place seed. Certainembodiments of the present invention can provide operators feedback onreal-time spacing and depth allowing downforce adjustments according toactual soil conditions.

One or more embodiments of the present invention utilize computer visionand image-based technologies to automate information gatheringassociated with seed spacing and depth. Such information gathering alsoidentifies the precise coordinates of each seed through the use of GPSsystems. Image mosaicking is used to combine overlapping images usingcommon reference points within the images and create a single image witha wider field of view. Creating a mosaicked image can be done usingdirect or feature based algorithms (Gosh, et al, 2016, Jain et al, 2013and Fathima et al, 2013). Direct methods work by finding a consistentset of correspondence and calculating correlations between features inthe image using all pixels and usually are performed using a correlationmatrix (Renuka, 2016). This method is useful when mosaicking images withlarge regions of overlaps including small translations and rotations(Renuka 2016, Prados, et al, 2014, Jain et al, 2013), but it requirescomplex calculation (Fathima et al, 2013). Feature based methodsidentify distinct low-level features such as edges, corners, or pixelsbetween the two images (Gosh, et al, 2016) and match them together toform a global correspondence (Fathima et al, 2013). This process reducesthe computational complexity (Renuka, 2016) and usually handle imageswith small regions of overlap (Renuka, 2016 and Jain et al, 2013), anddetection of common features is possible even at changing geometricviewpoints (Gosh, et al, 2016).

Automating the process of measuring seed spacing, depth, and geolocationprovides significant information on the location of the seeds in thefurrow, which can be used to improve planter performance and implementoptimal planter settings. Thus, the present invention generally providesa system to measure real-time seeding depth, seed spacing, and seedlocalization during planting. Specifically, embodiments of the presentinvention aim to accomplish at least one of the following: (1) stitchcaptured real-time images of individual seeds planted, (2) measure seedspacing using the stitched image, (3) record actual seeding depth duringplanting, and (4) provide GPS coordinates of individual images.

In certain embodiments of the present invention, the measurement of seedplacement uses a high-speed camera, a light section sensor, and a GPSunit. FIG. 1 illustrates an embodiment of a seed planting assembly 10.Assembly 10, as depicted, comprises a seed planting device 12, namely aplanter row unit, a plurality of which can be attached to a commonframework to form a multirow planter capable of simultaneously plantingany desired number of rows. The seed planting device 12 is configured tocreate a furrow 14 in the ground 16, deposit a seed 18 into the furrow,and close the furrow. The furrow 14 is created by one or more openers20, such as opening discs. The planter row unit 12 may also comprise oneor more gauge wheels 22 that are operable to control the depth of thefurrow 14 created by the one or more seed openers 20. Seeds 18 areinitially held in seed bin 24 and metered therefrom to be deposited inthe furrow 14, using, for example, a seed tube (not illustrated). Theunit 12 is connected to the implement header 26 via linkage arms 28.

The unit 12 comprises a camera 30 that is configured to capture an imageof each seed 18 in the furrow 14 before the furrow can be closed byclosing wheels 32. The camera 30 is mounted in a downward configurationso that the camera lens faces the furrow 14 where seeds 18 areapproximated to drop from a seed tube of the planter. The unit 12further comprises a GPS unit 34 that is located in vertical alignmentwith the camera 30, and particularly the camera lens, as indicated byaxis A. The GPS unit 34 is operable to detect the geographic coordinatesof the center of an image of the seed 18 within furrow 14. The assembly10 further includes a processor that is operable to analyze consecutiveimages captured by camera 30 and to stitch the images together therebyforming a stitched image (see, e.g., FIG. 5) comprising at least twoadjacent seeds 18 within the furrow 14.

In one or more embodiments, camera 30 is positioned on planter row unit12 in between the one or more gauge wheels 22 and the one or moreclosing wheels 32, although this need not always be the case and maydepend upon the configuration of the planter row unit. In one or moreembodiments, the GPS unit 34 is mounted on the planter row unit 12directly over the camera 30 so that it is capable of receiving GPS radiosignals without obstruction from assembly 10.

In one or more embodiments, the assembly 10 further comprises a lightsection sensor 36 positioned to be in facing relationship to the furrow14 and operable to detect the deepest portion of the furrow 14 containedwithin the image captured by the camera 30 and the ground 16 adjacent tothe furrow 14. In one or more embodiments, the light section sensor 36is operable to calculate the difference between the deepest portion 38of the furrow 14 and the ground 16 adjacent to the furrow 14. Thisdifference corresponds to a depth of the seed 18 in the ground 16. Inone or more embodiments, the seed planting assembly 10 further comprisesa potentiometer 40 mounted thereon that is configured to provideinformation to the processor corresponding to vertical displacement ofthe seed planting device 12 during planting operations and to calculatea unit increase or decrease in the measuring position of the lightsection sensor 36. In one or more embodiments, assembly 10 furthercomprises a light source 42 mounted on the seed planting device 12 thatis configured to illuminate the furrow 14 and seeds 18 during imagecapture by the camera 30. In certain embodiments, the light source 42comprises one or more LEDs.

In one or more embodiments, assembly 10 can be used to measure theplanting characteristics of seeds 18. In such methods, furrow 14 iscreated in the ground 16 and a first seed 18 is deposited within thefurrow. Camera 30 is used to capture a first image that comprises thefirst seed 18 within the furrow 18. GPS unit 34, which is positioned invertical alignment with the cameral 30, detects and records thegeographic coordinates of the center of the first image. Next, as theassembly 10 progresses forward in the field, a second seed 18 isdeposited within the furrow 14. Camera 30 is used to capture a secondimage that comprises the second seed 18 within the furrow 14. The GPSunit 34 detects and records the geographic coordinates of the center ofthe second image. The processor is used to stitch the first and secondimages together to formed a stitched image that comprises the first andsecond seeds 18. The stitched image is then analyzed, using, forexample, the processor, to determine the number of pixels between thefirst and second seeds 18 and to convert the number of pixels into abasic unit of measurement using a pixel-to-distance calibration factorthereby determining the spacing between the first and second seeds 18.

In one or more embodiments, the method comprises using the geographiccoordinates of the centers of the first and second images to determinethe geographic coordinates of the first and second seeds 18. Also, inone or more embodiments, light section sensor 36, which is positioned infacing relationship to the furrow 14, is used to detect the deepestportion 38 of the furrow 14 contained within the first and second imagescaptured by the camera 30 and the ground 16 adjacent the furrow todetermine a depth of the seed 18 in the ground. The potentiometer 40,which is mounted on device 12 that is creating furrow 14 and depositingseeds 18, can be used to provide information to the processorcorresponding to a vertical displacement of the seed planting device 12during planting operations and to calculate a unit increase or decreasein the measuring position of the light section sensor 36. The system isprogrammed to capture images and collect GPS coordinates simultaneouslyat equal sampling frequency. For example, digital camera 30 can becapable of capturing 700 fps. However, capturing all of these imageswould generate needless volumes of information as only one imagecontaining each seed is necessary. Therefore, advantageously, the camera30 can be programmed to capture images based upon the forward velocityof the planting device 12 during planting operations. It is common forplanting equipment to be used at a forward velocity of between 3 to 10mph, with 6 mph being particularly preferred. In that range of forwardvelocity, the camera 30 can be operated to capture between 10 to 20 fps.Preferably, each image captured by the camera 30 will contain an imageof at least one seed 18. In a certain minority of images, two seeds willbe captured in the image. However, this does not present a problem asthe image can be stitched with immediately preceding or following imagesand the planting characteristics of both seeds can be analyzed.

The following description is directed toward an exemplary apparatuswhich several principles of the present invention were tested. Acultivation test apparatus comprising rails upon which the speciallyconfigured planter is mounted was used. To measure seed spacing, the SAMsystem captures images as seeds drop into the furrow.

Preferably, nearly every image contains one seed, and the SAM systemstiches two consecutive images with one seed in each image (see, FIGS. 4and 7A and 7B) to create a stitched/single image which contains twoseeds (see, FIGS. 5 and 7C). The system utilizes this image to perform aspatial calibration where it converts pixels into real world units.After calibration, the system repeats the same process using twodifferent successive images, and then uses the calibration value tomeasure the seed spacing from the generated stitched image. To measureseeding depth, the light section sensor, which is mounted along thecamera also facing the furrow, calculates the difference between thelowest part of the furrow and the ground. This difference is the seedingdepth or depth of the furrow. FIG. 9 shows actual seeding depth comparedto recorded or measured seeding depth using the system. FIG. 10B showsthe GPS coordinate of one image (FIG. 10A) provided by the GPS unit.

Seed Spacing Measurement

In one embodiment of the present invention, the process of calculatingthe seed spacing comprises the following steps:

(1) Image acquisition

(2) Image stitching

(3) Spatial calibration

(4) Calculation of seed spacing

Image Acquisition

Exemplary apparatus for image capture that was assembled and testedincluded a high-speed camera (acA640-750uc, Basler AG, Ahrensburg,Germany) configured using the NI Measurement and Automation Explorer(MAX) installed in LabVIEW (National Instrument, Austin, Tex., USA). Thecamera was connected to a control laptop computer (Latitude 14 3470,Dell, Round Rock, Tex., USA) with a 2.5 GHz Intel Core i7-6500U CPU(Intel, Santa Clara, Calif., USA) and 8 GB installed memory (RAM))through the USB 3.0 interface. The camera exposure time was set at 488microseconds (μs) to prevent capturing blurred objects or features onthe images. Since the amount of light is proportional to the exposuretime, an LED strip tape (4NFLS-x2160-24V, SBL, St. Louis, Mo., USA) wasused to provide additional lighting to illuminate features or objects ofinterest on the ground. The camera was mounted in between the gaugewheels and the closing wheels at a vertical distance of 8 inches fromthe camera lens to the ground level. The camera was fitted with a 5Megapixel C-Mount fixed focal lens (C125-0418-5M, Basler AG, Ahrensburg,Germany) which provided a field of view (FOV) corresponding to an imagesize of 15.7 cm by 11.7 cm. The camera was oriented such that the lensfaced the furrow where seeds are assumed to drop from the seed tube andaligned vertically with the sub-inch accuracy, real-time kinematic (RTK)GPS unit (GR5, Topcon Positioning Systems, Inc., Livermore, Calif., USA)mounted on top of the row unit. See, FIG. 1.

To ensure the camera captured more than 50% overlap on the images foreffective image stitching, the high-speed camera was configured totransmit and record at 10 fps at a bit rate of 92 MB/s over a USD 3.0interface using the Pylon Viewer (Basler AG, Ahrensburg, Germany). Imageresolution was about 0.3 Megapixel with pixel dimension of 656×496pixels. Likewise, the Horsch Terminal ME controller (Horsch LLC,Mapleton, North Dakota) was programmed to plant corn at 103,200seeds/hectare seeding rate that corresponds to a seed spacing of 12.7cm. The row unit was mounted on the customized cultivation assessmenttest apparatus (CAT App), which comprised a row unit toolbar that can beraised/lowered and moved back/forth along the 12.2-m long rails by a 31HP gasoline engine (Vanguard, Briggs and Stratton, Wauwatosa, WI). Afour-wheel tractor (LA1251, Kubota, Grapevine, Tex.) was used to pulland move the CAT App within the field during testing.

A separate program controls the speed of the engine which was programmedto run the set up at a target speed of 6.4 kph for all the tests. Duringtesting, the closing wheels of the row unit were raised to prevent itfrom closing the furrows. This enabled the manual measurement of seedspacing that was used for comparing the actual and calculated seedspacing using the root mean square error (RMSE) equation. This is ameasure of how close the calculated spacing is to the actual spacing andis represented by as equation 1.

$\begin{matrix}{{RMSE} = \sqrt{\frac{\sum\limits_{i = 1}^{n}( {{\overset{\hat{}}{y}}_{i} - y_{i}} )^{2}}{2}}} & (1)\end{matrix}$

Image Stitching

A feature-based matching algorithm was used to combine captured imagesto create a panoramic image for seed spacing measurement. The scaleinvariant feature transform (SIFT) algorithm is an effective tool toextract common feature points and perform matching between two imageswith significant overlap and invariant to noise, occlusion andillumination changes. The matching algorithm developed in MATLAB(R2017a, Natick, Mass., USA) was used to find corresponding pointsbetween the reference image and the image to be matched. There are fivesteps on how the algorithm is implemented as outlined by Ghosh andKaabouch (2016) and Lowe (2004). These are scale-space construction,scale space extrema detection, keypoint localization, orientationassignment and keypoint descriptors. The first step involves theconstruction of scale space by generating several octaves or blurredimages from the input image by applying a Gaussian filter or Gaussianblur operator to reduce noise and image details. Mathematically, thiscan be expressed by equation 2 as defined by Lowe (2004).

L(x, y,σ)=G(x, y, σ)*I(x,y)   (2)

where L(x, y, σ) is the blurred image, * is the convolution operator,G(x, y, σ) is the Gaussian blur operator and I (x, y) is the inputimage. Next step was detecting key feature points in the scale spaceusing a difference-of-Gaussian (DoG) operation by calculating thedifference of two adjoining blurred images, L, using equation 3 asdefined by Lowe (2004).

D(x,y,σ)=G(x,y,kσ)−L(x,y)   (3)

where k is a constant multiplicative factor. Keypoint candidates in astack of DoG images are detected by comparing a pixel to its neighboringpixels at the current and adjacent scales. See, FIG. 2. This processgenerated low contrast keypoints or extrema located on an edge which arethen eliminated to improve matching efficiency of the algorithm.

Assigning an orientation for the keypoint is done to provide rotationinvariance. This process was done by assigning the dominant orientationto the keypoint based on gradient directions and magnitude around it.The orientation, θ (x, y), for each image, L(x,y), is calculated usingequation 4 defined by Lowe (2004).

θ(x,y)=arctan((L(x,y+1)−L(x,y−1))/(L(x+1,y)−L(x−1,y)))   (4)

This procedure resulted in an orientation histogram where dominant localgradient orientations were identified and used to create a keypoint withthat orientation. The last step is computing a descriptor or afingerprint of the keypoint to differentiate it from other keypointsgenerated.

Recognizing distinct features or objects in an image is performed byfirst matching each feature or keypoint independently to the database ofkeypoints extracted from a reference image. However, many of theseinitial matches can be incorrect due to some outliers orindistinguishable features that arise from background noise in theimage. Thus, a random sample consensus (RANSAC) algorithm was used toremove false matches or outliers and created a transformation orhomography matrices that was used to stitch two overlapping imagesproducing a stitched image.

Spatial Calibration

After generating the stitched image, a process called simple spatialcalibration was performed to determine the relation of image pixels toreal-world units. The spatial calibration process is illustrated in FIG.3. The first step 44 in this process comprises forming the stitchedimage as described above. The image is then plotted 46 and a linecreated 48 that connects the two seeds in the image. By using an imagerywith two seeds of known spacing or distance 50, this distance in pixelswas calculated by a spatial calibration algorithm developed in MATLABusing the Euclidean distance formula as shown in equation 5.

d=√{square root over ((x ₂ x ₁)²+(y ₂ −y ₁)²)}  (5)

where d is the number of pixels between the two objects in the image,(x₁, y₁) is the coordinate of the first object and (x₂, y₂) is thecoordinate of the second object. The derived conversion factor 52 fromthe spatial calibration was then added in the seed spacing algorithmthat was used in the calculation of the seed spacing.

Calculate Seed Spacing

Once spatial calibration was done, two succeeding images were stitched,and the seed spacing was measured using the seed spacing algorithm. Thealgorithm calculates the spacing in pixels then multiplied to thecalibration factor. This process was done independently for eachstitched image.

Seeding Depth Measurement

A light section sensor (OH7-Z0150.HI0720.VI short, Baumer Electric AG,Frauenfeld, Switzerland) was used to record the seeding depth. Thesensor was designed to measure the height difference between the lowestand highest point on the ground using a laser. It was attached to the 3Dprinted frame that was placed between the gauge wheels and the closingwheels along the center of the furrow.

A potentiometer (model 424A11A090B, Elobau sensor technology, Inc.,Waukegan, Ill., USA) with a linear response of 4 to 20 mA and 12 mA asthe center position was mounted on the row unit to provide informationon row unit vertical movement or measuring position displacement. Alaboratory set up was constructed to develop a relationship curvebetween the light section sensor and the potentiometer by recordingvarying depth measurements at changing potentiometer positions using 12mA as the reference position. This data was plotted in SAS Universityedition to generate a calibration equation where seeding depth was theresponse variable and the measuring position as the predictor variable.The slope represents the amount of change for every unit increase ordecrease in the measuring position. During in-field test experiments,the recorded position was subtracted from the reference position andthen the difference multiplied by the slope of the line to get thechange in seeding depth due to the position displacement. The actualdepth was then calculated by subtracting the calculated depth from thechange in seeding depth.

Measurement System Set Up

The developed system used in the described experiments to measure seedspacing, depth and geo-location of corn seed comprises a high-speedcamera, light section sensor, LED light strip, potentiometer, GPS, dataacquisition system and a control computer. The system comprises twoseparate LabVIEW programs collecting data at 10 Hz: (1) a program torecord seeding depth and location which outputs data in a .txt file, and(2) the imaging program which outputs data in a .jpg file. As the systemis initiated, the high-speed camera captures images, the light sectionsensor records seeding depth, and the GPS unit acquires geo-locationssimultaneously, all of which is saved onto an external hard drive(Transcend, Orange, Calif.). Thus, each image comprises data on seedingdepth and geo-location.

Seed Spacing

The test location was a no-till field with a volumetric water contentduring testing ranging from 18% to 20.8%, averaging 19.1%. After eachtest run, actual seed spacing was manually measured by laying ameasuring tape along the row beside the furrow. This data was later usedto calculate the error in the seed spacing measured using the system.

Samples of two successive images with overlap used in spatialcalibration are shown in FIGS. 4A and 4B. FIG. 4A is the referenceimage, and FIG. 4B is the target image. The overlap between the twoimages is shown by the dashed line.

The image stitching algorithm used this overlap as the matching windowto extract common features to determine the correspondence between thetwo images before combining them into one single or stitched image (FIG.5). After generating the stitched image, the spatial calibration wasperformed resulting in a pixel-to-actual distance calibration factorconverting a pixel into basic unit of measurement. This procedureresulted in a calibration factor of 0.022 cm per pixel. See, FIGS. 6Aand 6B.

Another set of two successive images was stitched (see, FIGS. 7A, 7B,and 7C) to calculate the seed spacing (FIGS. 8A and 8B).

The results of the field tests showed the seed spacing measured usingthe system and manual measurements are presented in Table 1. The rootmean square error (RMSE) was used to measure the system accuracy and theestimated or calculated spacing were regressed to actual spacingmeasurements to determine the models' coefficient of determination (R²).Overall, the system was able to achieve an RMSE of 0.63 cm and an R² of0.87. Measurement errors shown by the residuals can be caused by severalfactors which consists of distortion of acquired images caused by thecamera lens and potential human errors during manual measurement ofactual seed spacing.

TABLE 1 Seed spacing test results (RMSE = 0.63 and R² = 0.87). Actualseed Distance, Conversion factor, Calculated seed Residual, spacing, cmpixel cm/pixel spacing, cm cm 12.7 425 0.028 12.0 0.7 11.9 410 0.02811.5 0.4 14.5 490 0.028 13.7 0.8 10.7 370 0.028 10.4 0.3 12.7 425 0.02811.9 0.8 11.7 410 0.028 11.5 0.2 10.9 420 0.028 11.8 −0.9 11.4 420 0.02811.8 −0.4 14.7 480 0.028 13.4 1.3 13.2 460 0.028 12.9 0.3 13.2 490 0.02813.7 −0.5 10.4 350 0.028 9.8 0.6 12.4 450 0.028 12.6 −0.2 8.9 325 0.0289.1 −0.2

Seeding Depth

FIG. 9 shows the recorded and measured seeding depths during theexperiment.

Recorded seeding depth are sensor measurements while measured seedingdepths are actual seeding depths taken during the experiment. Overall,recorded seeding depths were within the tolerance of +/−6.5 mm frommeasured seeding depth which suggests the system has the capability ofmeasuring real-time seeding depths accurately. This tolerance is lowerthan the capability required to monitor real-time seeding and will allowoperators to adjust gauge wheel load levels accordingly. This willprevent over and under application of load during planting which couldpotentially reduce areas of shallow seeding depth or sidewallcompaction.

Image GPS Coordinates

FIG. 10 shows an image of a seed deposited in a furrow and thecorresponding GPS coordinate for the seed. This coordinate can be used,for example, for analysis of missing plants. In certain embodiments, thesystem may utilize this information to automatically geotag each image.In certain embodiments, one seed may be shown in multiple images due tothe programmed degree of overlap. However, since the target distancebetween seeds can be very narrow, sometimes one GPS coordinate is thesame for two images. Therefore, visual inspection of individual imagesmay be performed to locate similar seeds to allow accurate assignment ofGPS coordinate for seed localization. To illustrate, refer to FIGS.7A-7C where two seeds can be seen. The images of FIGS. 7B and 7C havethe same GPS coordinates which can be used to locate seed 2. On theother hand, for an image a where two seeds are present, the GPScoordinate of that image can be used to locate seed 1 since seed 2 wasalready assigned with its own GPS coordinate.

The ability to locate seeds provides the ability to gather informationon the cause of wide gaps after seed emergence. GPS locations of areaswith wide gaps can be collected and matched to the recorded GPScoordinates, which can be used to confirm placement of seed. Thepresence of a seed in between plants indicate proper seed metering ofthe planter and non-emergence can be due to seed germination issues orsome other factors. For example, in areas where plants did not emergeevenly, finding the GPS coordinate of gaps between emerged plants thenmatching it to the collected coordinates of images with the plantedseeds allows the growers to determine if there was a seed planted or ifthe seed did not emerge. The results can provide data on plantermetering performance or germination issues.

CONCLUSION

The results demonstrate that the systems according to the presentinvention have the ability to be used for measuring seed spacing andseeding depth, of corn especially, on row crop planters. These two seedplacement parameters are important to growers as it allows them todetermine final plant population and potential yield. Having the abilityto understand real-time seeding depth and spacing allows operators toadjust planter settings on the go. At present, seeding depth is measuredmanually by digging individual plants locating the seed then measure thedistance to the ground. This process requires a lot of manpower and isprone to errors. Moreover, simply digging a couple of plants may not beenough to correctly assess an entire field as it requires multiplestrips equivalent to hundreds of plants to represent the seeding depthfor the whole field. Likewise, plant spacing is currently being measuredby laying measuring tape and recording the cumulative spacing. Usually,plant spacing and seed spacing are used interchangeably as seeds areassumed to have been spaced uniformly during planting. However, in somecases where non-emergence due to soil compaction and non-germination ofseeds occur, seed spacing and plant spacing could be different. Thus,one way to understand planter performance is to measure both seedspacing and plant spacing. The developed system is capable of capturingreal-time images of seeds planted during planting and can be used togenerate a stitched image of successive images. The generated stitchedimage was used to calculate seed spacing where it resulted in an RMSE of0.63 and an R² of 0.87. Likewise, recorded depth was within a toleranceof +/−6.5 mm from measured seeding depth suggesting the system canmeasure real-time seeding depths accurately. Furthermore, the system wasable to record GPS coordinates of individual images which can be used tomap planted seeds. The data collected allows real-time measurement ofthe singulation performance, seed placement accuracy, and seed location,which may be used to optimize planter performance across the field andlead to more uniform plant stand and population, and improved yield.

REFERENCES AGCO Corp. (2015). The art and science of plantingperformance. Retrieved fromhttp://www.white-planters.com/assets/pdf/white-planters-the-art-and-science-of-planting-performance.pdf

Alessi, J. & Power, J. F. (1971). Corn emergence in relation to soiltemperature and seeding depth. Agronomy Journal, Vol. 63.

Barbedo, A. J. (2013). Digital image processing techniques fordetecting, quantifying and classifying plant diseases. SpringerPlus,2(1), 1-12. doi:10.1186/2193-1801-2-660

Blasco. J., Jose V. Benlloch, Manuel Agusti, Enrique Molto. (1999).Machine vision for precise control of weeds. Proc. SPIE 3543, PrecisionAgriculture and Biological Quality. http://dx.doi.org/10.1117/12.336897

Doerge, T., T. Hall, D. Gardner, 2002. New research confirms benefits ofimproved plant spacing sensing in corn. Crop Insights Vol. 12, No.2.Pioneer Hi-Bred Int'l.

Ghosh, D. & Kaabouch, N. (2016). A survey on image mosaickingtechniques. Journal of Visual Communication and Image Representation. 34(1-11). https://doi.org/10.1016/j.jvcir.2015.10.014

Hetzroni, A., G. E. Miles, B. A. Engel, P. A. Hammer R. X. Latin (1994).Machine vision monitoring of plant health. Advances in Space Research.14(11). https://doi.org/10.1016/0273-1177(94)90298-4

Jain, P. M. & Shandliya, V. K. (2013). A review paper on variousapproaches for image mosaicing. Int. Journal of ComputationalEngineering Research. Vol 3(4).

Kacira, M. & P. P. Ling. (2001). Design and development of an automatedand non-contact sensing system for continuous monitoring of plant healthand growth. Trans. ASAE, 44(4), 989-996. doi:10.13031/2013.6231

Kim Chim B., Omara, P., Macnack, N., Mullock, J., Dhital, S. & Raun, W.(2014). Effect of seed distribution and population on maize grain yield.Int. Journal of Agronomy. http://dx.doi.org/10.1155/2014/125258.

Krall, J. M., H. A. Esechie, R. J. Raney, S. Clark, G. TenEyck, M.Lundquist, N. E. Humburg, L. S. Axthelm, A. D. Dayton, and R. L.Vanderlip (1977). Influence of Within-row Variability in Plant Spacingon Corn Grain Yieldl. Agron. J. 69:797-799.doi:10.2134/agronj1977.00021962006900050016x

Knappenberger, T. & Koller, K. (2012). Spatial assessment of thecorrelation of seeding depth with emergence and yield of corn. PrecisionAgric. 13:163-180. DOi 10.1007/s11119-011-9235-4.

Lowe, D. G. (2004). Distinctive image features from scale-invariantkeypoints. Int. J. Comput. Vis., 60 pp. 91-110

McCarthy, C. L., Hancock, N. H., & Raine, S. R. (2010). Applied machinevision of plants: A review with implications for field deployment inautomated farming operations. Intelligent Service Robotics, 3(4),209-217. doi:10.1007/s11370-010-0075-2

Nielsen R. L. (2001). Stand establishment variability in corn. Dept. ofAgronomy publication #AGRY-91-01. Retrieved fromhttps://www.agry.purdue.edu/ext/pubs/AGRY-91-01_v5.pdf

Nakarmi, A. D., & Tang, L. (2012). Automatic inter-plant spacing sensingat early growth stages using a 3D vision sensor. Computers andElectronics in Agriculture, 82, 23-31. doi:10.1016/j.compag.2011.12.011

Nejati, H., Azifimar, Z., & Zamani, M. (2008). Using fast fouriertransform for weed detection in corn fields. Int. Conf. on Systems, Manand Cybernetics. Oct. 12-15, 2008. DOI: 10.1109/ICSMC.2008.4811448

Prados, R. (2014). Image Blending Techniques and their Application inUnderwater Mosaicing. Springer Briefs in Computer Science, DOI:10.1007/978-3-319-05558-9_2.

Renuka, D. (2016). Image mosaicing using phase correlation and featurebased approach: A review. International Journal of Engineering Research,4 (1).

Sanchiz, J. M., Pla, F. Marchant, J. A & Brivot, R. (1996). Structurefor motion techniques applied to crop field mapping. Image and VisionComputing, 14.p.353-363.

Shrestha, D. S., Steward, B. L., & Birrell, S. J. (2004). Videoprocessing for early stage maize plant detection. BiosystemsEngineering, 89(2), 119-129. doi:10.1016/j.biosystemseng.2004.06.007

Shrestha, D. S. & B. L. Steward. (2003). Automatic corn plant populationmeasurement using machine vision. Trans. ASAE, 46(2), 559.doi:10.13031/2013.12945

Tang, L. & L. F. Tian. (2008). Real-time crop row image reconstructionfor automatic emerged corn plant spacing measurement. Trans. ASABE,51(3), 1079-1087. doi:10.13031/2013.24510

Tang, L., L. Tian, & B. L. Steward. (2000). Color image segmentationwith genetic algorithm for in-field weed sensing. Trans. ASAE, 43(4),1019. doi:10.13031/2013.2970

Thomison, P., Jeschke, M., & Butzen, S. (2012). Planting depth effectson corn stand and grain yield. Retrieved fromhttps://www.pioneer.com/home/site/us/agronomy/library/planting-depth-and-stand-yields

USDA-ERS (2017). 2017 Farm sector income forecast. Retrieved fromhttps://www.ers.usda.gov/topics/farm-economy/farm-sector-income-finances/farm-sector-income-forecast/

USDA(2017a). World agricultural supply and demand estimates. Retrievedfrom https://www.usda.gov/oce/commodity/wasde/latest.pdf

USDA (2017b). USDA Agricultural projections to 2026. Retrieved fromhttps://www.ers.usda.gov/webdocs/publications/82539/oce-2017-1.pdf?v=4278

1. A seed planting assembly comprising: a seed planting deviceconfigured to create a furrow in the ground, deposit a seed into thefurrow, and close the furrow; a camera attached to the seed plantingdevice and configured to capture an image of the seed in the furrowprior to closing of the furrow; a GPS unit attached to the seed plantingdevice and located in vertical alignment with the camera and operable todetect the geographic coordinates of the center of the image; and aprocessor operable to analyze consecutive images captured by the cameraand to stitch the images together thereby forming a stitched imagecomprising at least two adjacent seeds within the furrow.
 2. The seedplanting assembly of claim 1, wherein the seed planting device comprisesa planter row unit.
 3. The seed planting assembly of claim 2, whereinthe planter row unit comprises one or more seed openers operable tocreate the furrow, one or more gauge wheels operable to control thedepth of the furrow created by the one or more seed openers, one or moreseed tubes operable to deposit the seed into the furrow, and one or moreclosing wheels operable to close the furrow following deposit of theseed in the furrow, the camera being positioned on the planter row unitin between the one or more gauge wheels and the one or more closingwheels.
 4. The seed planting assembly of claim 3, wherein the GPS unitis mounted on the planter row unit directly over the camera.
 5. The seedplanting assembly of claim 1, wherein the assembly further comprises alight section sensor positioned to be in facing relationship to thefurrow and operable to detect the deepest portion of the furrowcontained within the image captured by the camera and the groundadjacent to the furrow, the light section sensor further being operableto calculate the difference between the deepest portion of the furrowand the ground adjacent to the furrow, the difference corresponding to adepth of the seed in the ground.
 6. The seed planting assembly of claim5, wherein the assembly further comprises a potentiometer mounted on theseed planting device and configured to provide information to theprocessor corresponding to vertical displacement of the seed plantingdevice during planting operations and to calculate a unit increase ordecrease in the measuring position of the light section sensor.
 7. Theseed planting assembly of claim 1, wherein the assembly furthercomprises a light source mounted on the seed planting device andconfigured to illuminate the furrow and seeding during image capture bythe camera.
 8. The seed planting assembly of claim 7, wherein the lightsource comprises one or more LEDs.
 9. A method of measuring plantingcharacteristics of seeds comprising: creating a furrow in the ground anddepositing a first seed within the furrow; using a camera to capture afirst image comprising the first seed within the furrow; using a GPSunit positioned in vertical alignment with the camera to detect andrecord the geographic coordinates of the center of the first image;depositing a second seed within the furrow; using the camera to capturea second image comprising the second seed within the furrow; using theGPS unit to detect and record the geographic coordinates of the centerof the second image; using a processor to stitch the first and secondimages together to form a stitched image, the stitched image comprisingthe first and second seeds; analyzing the stitched image to determinethe number of pixels between the first and second seeds and to convertthe number of pixels into a basic unit of measurement using apixel-to-distance calibration factor thereby determining the spacingbetween the first and second seeds.
 10. The method of claim 9, whereinthe method further comprises using the geographic coordinates of thecenters of the first and second images to determine the geographiccoordinates of the first and second seeds.
 11. The method of claim 9,wherein the method further comprises: using a light section sensorpositioned in facing relationship to the furrow to detect the deepestportion of the furrow contained within the first and second imagescaptured by the camera and the ground adjacent to the furrow; andcalculating the difference between the deepest portion of the furrow andthe ground adjacent to the furrow to determine a depth of the seed inthe ground.
 12. The method of claim 11, wherein the method furthercomprises: using a potentiometer mounted on a device that is creatingthe furrow and depositing the seeds to provide information to theprocessor corresponding to a vertical displacement of the seed plantingdevice during planting operations and to a unit increase or decrease inthe measuring position of the light section sensor.